A Theory of Lossy Compression for Individual Data

We develop rate-distortion theory for individual data with respect to general distortion measures, that is, a theory of lossy compression of individual data. This is applied to Euclidean distortion, Hamming distortion, Kolmogorov distortion, and Shannon-Fano distortion. We show that in all these cases for every function satisfying the obvious constraints there are data that have this function as their individual rate-distortion function. Shannon’s distortion-rate function over a random source is shown to be the pointswise asymptotic expectation of the individual distortion-rate functions we have defined. The great differences in the distortion-rate functions for individual non-random (that is, the aspects important to lossy compression) data we established were previously invisible and obliterated in the Shannon theory. The techniques are based on Kolmogorov complexity.